1. Field of the Invention
The present invention relates to, for example, software for image segmentation/extraction in an image recognition system, a moving body detection system, a digital camera, a digital video camera, a robot vision, an authentication system by means of facial recognition, a security system, an artificial intelligence (AI) system, etc. and an image segmentation method, an image segmentation apparatus, an image processing method, and an image processing apparatus which are applicable to an image segmentation/extraction integrated circuit (IC).
2. Description of the Related Art
Recently, there is a desire for an increase in speed of image recognition in order to realize intelligent information processing technologies. For example, to realize an intelligent robot that behaves and makes a decision in nearly the same way as a human and real-time facial recognition or moving body recognition, it is necessary to speedily process visual information (information of a natural image) which is taken in from a camera etc. Especially in the case of control or image recognition for robots, the visual information needs to be processed on the fly. The visual information, however, is typically vast in amount and so it takes a considerably long time to process it using a general purpose computer, etc.
One of the fundamental and indispensable processing items required to perform image processing such as image recognition is image segmentation. This image segmentation is processing to take out an individual object (for example, a human face or a moving body such as a vehicle) from a complicated natural image which is taken in as an input, so that this processing is fundamental and indispensable in order to perform image processing such as image recognition. There have been made a number of proposals for image segmentation so far. Those image segmentation methods proposed are classified as follows.
(1) Method based on profile line
(2) Method based on region
(3) Combination of methods (1) and (2) and method which formulates logical expressions which optimize the combination
Method (1) is described in detail in the following references 1 and 2. Further, method (2) is described in detail in reference 1. Even further, combination method (3) is described in detail in the following reference 3.
Reference 1: J. C. Russ, “The image Processing Handbook”, CRC PRESS, (1999).
Reference 2: S. Sarker and K. L. Boyer, “Integration inference, and management of spatial information using Bayesian networks: Perceptual organization”, IEEE Trans. Pattern Anal. Machine Intel., Vol. 15, pp. 256-274, (1993).
Reference 3: S. M. Bhandarker and H. Zhang, “Image segmentation using evolutionary computation”, IEEE Trans. on Evolutionary Computation, Vol. 3, No. 1, (1999).
Of these image segmentation methods, method (2) based on the region is referred to as a region growth type one and attracting attention as the one that can segment an object accurately.
It is to be noted that the image segmentation methods proposed so far all premise that a color or grayscale image be processed by software. Therefore, these methods involving a complicated processing procedure and take much time in processing. To speed up this processing, preferably it is realized by hardware. A relevant algorithm, however, is complicated, making it difficult to realize the algorithm in a relatively small area. As a result, the algorithm cannot but rely on software, providing a present situation of an extreme difficulty in realization of real time processing (which takes a few seconds or so). Furthermore, the color and grayscale natural images each require one dedicated algorithm for their segmentation.
While on the other hand, for a binary image, a few hardware processing methods have been proposed so far which realize high speed labeling. See the following references for example.
Reference 4: E. Mozef et al., “Parallel architecture dedicated to image component labeling in 0(nlogn): FPGA implementation”, Proceedings of SPIE, Vol. 2784, pp. 120-125, (1996).
Reference 5: Y. Ishiyama et al., “Labeling board based on boundary tracking”, Systems and Computers in Japan, Vol. 26, No. 14, pp. 67-76, (1995).
Reference 6: Ishiyama et al., “Labeling board based on boundary tracking”, the Institute of Electronics, Information, and Communication Engineers Research Paper Magazine D-II, Vol. J78-D-II, No. 1, pp. 69-75, (1995).
Reference 7: S. D. Jean et al., “New algorithm and its VLSI architecture design for connected component labeling”, Proceedings of Int'l Symp. on Cir. &Sys. (ISCAS), Part 2 (of 6), pp. 565-568, (1994).
These methods, however, are dedicated for use in processing of binary images and handle only one-bit values for each pixel, which means that they cannot easily be applied directly to the processing of the color or grayscale natural image.
To date, D. L. Wang et al. have proposed an image segmentation algorithm for the grayscale image based on a cell network model LEGION (Locally Excitatory Globally Inhibitory Oscillator Network) (see Reference 8: D. L. Wang and D. Terman, “Image segmentation based on oscillator correlation”, Neural Computation, Vol. 9, No. 4, pp. 805-836, (1997).
In this model, cells are correlated with pixels of a segmentation-subject image, so that the image is segmented using the non-linear dynamics of the cell network, based on synchronous and asynchronous oscillation states of each of the cells. To realize this directly, however, it is necessary to solve a plurality of differential equations for each of the pixels, so that the image segmentation is carried out highly accurately, but is time consuming. Therefore, to realize real time processing, it is necessary to speed up the processing by realizing it by hardware.
To this end, there is proposed a method to utilize an analog circuit in order to realize the nonlinear dynamics of the cell network based on the LEGION for a grayscale image. See the following references for example.
Reference 9: H. Ando, T, Morie, M. Nagata, and A. Iwata, “Oscillator networks for image segmentation and their circuits using pulse modulation methods”, Procs. 5'th International Conference on Neural Information Processing (ICONIP'98), pp. 586-589, Kitakyushu, Oct. 21, (1998).
Reference 10: H. Ando, T. Morie, Nagata and A. Iwata, “A nonlinear oscillator network for gray-level image segmentation and PWM/PPM circuits for its VLSI implementation”, IEICE Trans. Fundamentals, Vol. E83-A, No. 2, pp. 329-336, (2000).
These methods by use of an analog circuit use a capacitor to store an analog quantity. A larger capacity requires a larger area of the capacitor, which inflicts a significant restriction on a decrease in area and an increase in operating speed of a future integrated circuit in an attempt to increase its integration density. Further, handling of an analog quantity is subject to an effect of fluctuations in a manufacturing process. Therefore, much attention must be paid during the manufacturing process, Making it not easy to realize the algorithm as an LSI chip even by state-of-the-art technologies.